1. Field of the Invention
This invention relates generally to computer testing, and more particularly to real-time XML based test report generation.
2. Description of the Related Art
Software applications typically undergo constant review and improvement, which verifies the functions of the software and updates the software to incorporate new features and functions. However, each new feature and error fix may cause addition errors. As such, test engineers generally apply a series of test cases, known as a test suite, to the application to test each new update and error fix. A typical test suite can include thousands of individual test cases.
For example, in a Java™ application, a test case generally includes an assertion, such as pushing a button, followed by an occurrence of a particular event. In addition, the test case validates the occurrence of the event, verifying whether the event occur correctly. For example, a test case can verify the function of a “tone” button on an Internet web page application. When the tone button is pressed, a tone should be produced. The test case simulates a computer mouse dragging a cursor to the button, pressing the button, and then releasing the button. The test case then validates whether the tone was correctly produced. Finally, a test result is output to a test report, which includes at least one test result for each one of the test cases.
Often, test reports are output in a text format so that software test engineers can review the test results in a written form. However, unformatted text-based test reports can be difficult for a human engineer to read. To make test reports easier to read, test suites often output the test reports in a Hypertext Markup Language (HTML) format, rather than an unformatted text format. HTML format allows a user to view the data using a browser application, which renders the data is easy for a human to read.
Conventional test applications typically use an HTML reporter to convert test results into HTML format. HTML surrounds data with tags that define how a browser should present the data to a user. Hence, an HTML test report file represents a view of the test result data, rather than the data itself. As such, an HTML test report file cannot easily be displayed in different views without severely altering the tags and other content of the HTML file. Thus, if a specific view of the test result data is needed, the new data view cannot be easily created without severely altering the HTML test report file.
Different users often require different views of the test report data. For example, a department manager may require a listing of test regressions, while an application designer may require a listing of all test failures. Using an HTML based test report file, these two data views generally cannot be easily created using the same HTML test report file. Creating these views typically requires the HTML test report file to be examined and altered to create new HTML files, which include only the information requested.
In view of the foregoing, there is a need for techniques that produce test report files in manner that can be viewed in multiple fashions without requiring substantial altering of the test report file. In addition, the methods should produce test report data in real-time, thus reporting test results as they occur to reduce the impact of system crashes that occur during application testing.